VMI: An Efficient Messaging Library for Heterogeneous Cluster Communication

Rapid advances in cluster interconnect technologies [2,14,15] that offer different capabilities has added a new dimension to the problem of managing heterogeneity in a clustered environment. Cluster users are now faced with the challenge of having to choose from among several communication APIs[4,9,10,11]. Recent research efforts have addressed the issues involved in supporting multiple communication methods to support heterogeneous devices[1,5]. Although these efforts provide a uniform API that can support communication across different devices in an application transparent manner at run time, they however, require applications to be recompiled when network devices are added or removed. Based on our experience administering a large scale production NT cluster at NCSA, we think a messaging library should not only support multiple communication methods, but should also allow communication devices to be added and deleted dynamically, without having to recompile applications and higher level messaging libraries. We also think that cluster administration can be simplified if a messaging layer has the ability to gracefully terminate a collective computation when its processes have failed. We have designed a high performance messaging API called Virtual Machine Interface(VMI), that incorporates these features and chooses from multiple underlying interconnects to deliver the best possible performance without any hints from the application. In short, VMI attempts to alleviate the burden of the programmer and cluster administrator when there is a wide choice of communication protocols and devices available.